Assessment of Vaccines

Slides: https://www.andreashandel.com/presentations/

Andreas Handel

2023-10-30

Vaccines are pretty good

xkcd.com

Evaluation of vaccines

How do we determine if vaccines are good?

Safety
Immunogenicity
Efficacy/Effectiveness
Cost-effectiveness

Vaccine Development

Knipe et al Science 2020

Outcomes of interest

Vaccines (partially) protect those who receive them (direct/individual effect):

Reduction in risk of infection/symptoms/hospitalization/death.
Reduction in strength of symptoms.

Outcomes of interest

Vaccines (partially) protect those who receive them (direct/individual effect):

Reduction in risk of infection/symptoms/hospitalization/death.
Reduction in strength of symptoms.

Vaccines can also protect non-vaccinated contacts (indirect effect).

Reduction of susceptibles in the population leads to overall reduced spread (contagion effect).
Reduction of infectiousness/transmission potential leads to reduced spread (infectiousness effect).

See Halloran & Hudgens 2016 CER and references therein.

Indirect effect example

Vaccine 1 reduces risk of clinical infection by 70%, reduces infectiousness by 30%.
Vaccine 2 reduces risk of clinical infection by 30%, reduces infectiousness by 70%.

Gallagher et al, medRxiv 2020

Ways to evaluate vaccine impact

Measure it:

Challenge studies
Clinical trials
Observational studies

Estimate it:

Correlates of Protection

Measuring vaccine impact

Challenge studies

One group receives the vaccine, the other placebo.
Both groups are challenged with the pathogen under consideration.
Measures vaccine efficacy (VE).
Well-controlled, can use small(ish) sample size.
Somewhat unrealistic (e.g., high challenge doses).
Direct effect only.
Sometimes not feasible/ethical.

Clinical trials

One group receives the vaccine, the other placebo.
Groups are followed and outcome (infection/disease/etc.) recorded.
Measures vaccine efficacy (VE).
Good balance between controlled and real-world setting.
Usually needed for FDA approval.
Only works if infections are high (not good for emerging pathogens).
Can measure direct and indirect effects (but usually only direct).
Expensive.

Observational studies

Taking vaccine is up to individuals (so must be licensed).
Cohort and case-control (e.g., test-negative) design.
Measures vaccine effectiveness (VE).
Most “real”, least controlled.
Can lead to biased estimates.
Can measure direct and indirect effects.
Can be fairly inexpensive.

Test-negative design

Sullivan et al 2014 Exp Rev Vac

Measuring vaccine impact - summary

Different study designs are available/useful.
As you learned from the Halloran & Hudgens paper, because of dependent happenings, designing and analyzing vaccine studies to properly capture all vaccine effects can be tricky.
Generally, a “classical” convincing phase 3 clinical trial is required to get approval (but see e.g., Ebola vaccine, H5N1 influenza vaccine).
Measuring actual outcomes is always expensive and time-consuming, sometimes not feasible (e.g., SARS-CoV-3 or H5N1 influenza vaccines).

Estimating vaccine impact

xkcd.com

Correlates of protection (CoP)

Determining an immunological quantity that correlates with protection can make vaccine assessment easier.
Finding correlates of protection (for vaccines) is very valuable (but can be tricky).

xkcd.com

Vaccine CoP terminology

It’s a mess.
Some individuals (e.g., Plotkin) mean by “correlate” a mechanistic/causal entity.
Some individuals mean by “correlate” something that correlates and might or might not be mechanistic/causal.
Other terms are used by some to try to be clearer (e.g., surrogate, mechanistic CoP). See e.g. Plotkin & Gilbert 2012 CID.
Protection is also often not clearly defined.

Correlates of protection

An “absolute correlate” (a la Plotkin) does not exist.
Levels at which a CoP leads to protection depend on pathogen, host, outcome, etc.

CoP - SARS-CoV-2 Example

Khoury et al 2021 Nat Med

CoP - SARS-CoV-2 Example

Khoury et al 2021 Nat Med

CoP - Influenza Example

Coudeville et al 2010 BMC MRM

CoP - Influenza Example

Age Group 50-64

CoP - Influenza Example

Age Group 65+

CoP - Influenza Example

Age Group 18-49

Estimating vaccine impact - summary

Using CoP can speed up approval process.
CoP can depend on details of vaccine (e.g., LAIV vs. IIV) and hosts (e.g., children vs. adults).
A full mechanistic understanding of
vaccine -> immune response -> protection
is still lacking for any vaccine (afaik).

Keep going?

Image by Aline Dassel/Pixabay

Research project - Assessing influenza vaccine candidates

Current Influenza vaccines

Need to be reformulated almost every year because of virus evolution
Need to be taken annually, due to virus evolution and vaccine waning
Are not very good, even if the vaccine and circulating strains match

Future Influenza vaccines

Should protect for a long time (lifelong?)
Should have high efficacy
Should protect against a wide range of strains

Universal flu vaccine challenges

Many
- How to assess/compare vaccine candidates

How do we define a vaccine response?

Quantifying vaccine responses

Magnitude: \(\frac{1}{N}\sum_{n} log(\textrm{TI}_{n,j=1})\)
Overall strength: \(\frac{1}{N*J}\sum_{n} \sum_{j} log(\textrm{TI}_{n,j})\)
Breadth: \(\frac{1}{N*J}\sum_{n} \sum_{j} \textrm{SC}_{n,j}\)

SC = Seroconversion, TI = Titer Increase (D28/D0), n = individuals, j = Strains.

Comparing vaccine responses

A new method to quantify/compare vaccine responses

Organize strains by antigenic distance
Fit a model to more robustly estimate magnitude/breadth/strength

Strain distance

Strain distance measures

Time: absolute difference in years of strain isolation.
Sequence: Some measure based on sequence difference.
Biophysical: Measures based on computed or measured biophysical properties.
Phenotypic: Antigenic cartography based on HAI assays.

“Our” data

Data from UGAFluVac study
Individuals received vaccine, response to multiple strains was tested

Strain distance measures

Quantifying vaccine responses

Comparing vaccine responses

Testing our method

We sampled from the panel of heterologous strains from UGAFluVac to mimic different labs

Testing our method - the results

The table shows the coefficient of variation for each outcome.

	Current method	Proposed method
Magnitude	0.088	0.103
Breadth	0.059	0.431
Overall strength	0.083	0.081

Our new method is worse (more variable)!

Testing our method with simulations

Create a universe of 50 possible heterologous strains with varying antigenic distances.
Create 10 lab panels by randomly sampling 9 strains and adding the homologous strain (distance of 0).
For each lab, generate 100 random individuals by simulating flu vaccine response titers from a model that shows linearly reduced response with increasing antigenic distance.

Simulation results

	Current method	Proposed method
Magnitude	0.025	0.008
Breadth	0.199	0.020
Overall strength	0.155	0.007

Now our new method is better. Hm…

The culprit

Simulation results with 30% censored data

	Current method	Proposed method
Magnitude	0.028	0.033
Breadth	0.290	0.316
Overall strength	0.137	0.071

With censored data, the current method looks artificially good.

Research Project Summary

Our proposed new method seems to be generally more robust.
If a good amount of censored data are present, the current method falsely under-estimates the uncertainty.
Our method also doesn’t properly handle the censored data (yet).
We need to update our method to properly deal with the censored values. Then we can do another comparison of our method and the current approach.

Wrap-up

Phase 3 trials are still the gold standard.
There is increased recognition that indirect effects can be important and should impact decision making.
Finding better CoP for any vaccine continues to be an important area of research.

Questions?

https://phdcomics.com/

Slides: https://www.andreashandel.com/presentations/
Contact: https://www.andreashandel.com